Customized augmented reality item filtering system

ABSTRACT

Examples provide customized augmented reality item filtering. An item filter analyzes item data using selection criteria and user preferences to identify high-interest item(s) and/or low-interest item(s) within a field of view (FOV) of a user device associated with a user. A high-interest item can include an item physically present within a portion of an item selection area or an un-stocked item not physically present within the portion of the item selection area. An augmented reality (AR) generator creates an AR image including a real-world image of the portion of the item selection area overlaid with graphical elements, including positive item indicators associated with high-interest items, negative item indicators associated with low-interest items, and/or un-stocked virtual items. The AR image is updated in real-time in response to user input, promotional data associated with one or more filtered item(s), and/or changes associated with the items within the FOV of the user device.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. patent application Ser. No.16/250,416, filed Jan. 17, 2019, which claims the benefit of U.S.Provisional Application No. 62/627,072, filed Feb. 6, 2018, thedisclosures of which are hereby expressly incorporated by reference intheir entirety.

BACKGROUND

Consumers typically prefer a large selection of items and options tochoose from when shopping within an item selection area. Consumers areaccustomed to selecting from a variety of brands, flavors, unit sizes,ingredients, price ranges, and other features. To locate a desired itemor type of item, consumers typically browse items on shelves, readinglabels, until the desired item or an acceptable substitute for the itemis located. However, this can be a time-consuming and inefficientprocess that can also be frustrating or overwhelming due to the numberitems on the shelves and varieties of those items. Moreover, consumersmay have difficulty identifying new items/unfamiliar items which wouldbe of interest or acceptable substitutes for desired items due to a lackof information about the new/unfamiliar items.

SUMMARY

Some examples provide an augmented reality system for filtering items inreal-time. The system includes a memory and at least one processorcommunicatively coupled to the memory. A filter component analyzes itemdata associated with a plurality of items within a field of view (FOV)of a user device associated with a user in an item selectin area. Theitem data is analyzed using a set of weighted selection criteria and aset of user preferences. The filter component selects a set ofhigh-interest items and a set of low-interest items for the user basedon the analysis. The set of high-interest items includes a set of itemsphysically available within the FOV of the user device and a set ofitems physically absent from the FOV of the user device. An overlaycomponent assigns a positive item indicator to each item in the set ofhigh-interest items. The overlay component assigns a negative itemindicator for each item in the set of low-interest items. An augmentedreality (AR) generator outputs an AR display of the FOV of the userdevice. The AR display includes a real-world image of a portion of theitem selection area within a FOV of the user and an AR overlay. The ARoverlay includes a set of positive item indicators associated with eachitem in the set of high-interest items, a set of negative itemindicators associated with each item in the set of low-interest items,and a virtual display comprising at least one un-stocked virtual itemphysically absent from the FOV of the user device.

Other examples provide a computer-implemented method for augmentedreality item filtering. An analysis component identifies a location of auser device associated with a user and a plurality of items within apredetermined distance of the user device based on the identifiedlocation and a site-specific planogram. A filter component identifies aset of high-interest items and a set of low-interest items from theplurality of items within the predetermined distance of the user. Aprediction component identifies a set of un-stocked items of potentialinterest to the user. Each item in the set of un-stocked items isphysically absent from a set of displays within the FOV of the user. Anoverlay component assigns a positive item indicator to each item in aset of high-interest items, a negative item indicator for each item inthe set of low-interest items, and a graphical representation for eachitem in the set of un-stocked items. An AR display generator generatesan AR display of a portion of an item selection area within a FOV of theuser. The AR display is sent to the user device for display to the user.The AR display includes a real-world image of a portion of the itemselection area within the FOV of the user enhanced by an AR overlay. TheAR overlay includes a set of positive item indicators associated witheach item in the set of high-interest items, a set of negative itemindicators for each item in the set of low-interest items, and agraphical representation of each item in the set of un-stocked items.

Still other examples provide a system for augmented reality itemfiltering. The system includes a memory; at least one processorcommunicatively coupled to the memory; and a set of sensors associatedwith a user device located within an item selection area. The set ofsensors generate sensor data associated with a plurality of items withinthe item selection area. An analysis component analyzes the sensor datausing item data and a planogram to identify a location of a user withinthe item selection area and identify a plurality of items within apredetermined distance of the user. A filter component analyzes theidentified plurality of items using a set of selection criteria and userpreferences. The filter component selects a high-interest category ofitems of potential interest to the user and a low-interest category ofitems within the predetermined distance of the user. An overlaycomponent assigns a positive category indicator to each item in theselected high-interest category and a negative category indicator toeach item in the selected low-interest category. An AR generatordisplays an AR display of the FOV to the user. The AR display includes areal-world image of a portion of the item selection area within a FOV ofthe user and an AR overlay enhancing the real-world image. The ARoverlay includes a positive category indicator associated with each itemin the selected high-interest category within the FOV. The AR overlayalso includes a negative category indicator for each item in theselected low-interest category within the FOV. The AR generator updatesthe AR display in real-time to change an indicator associated with anidentified item in the selected low-interest category from a negativecategory indicator to a positive category indicator on condition apromotional offer associated with the identified item is received from aset of suppliers.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an exemplary block diagram illustrating a system forcustomized item filtering using augmented reality (AR).

FIG. 2 is an exemplary block diagram illustrating a system for per-usercustomized filtering of items via a user device.

FIG. 3 is an exemplary block diagram illustrating an item selectionarea.

FIG. 4 is an exemplary block diagram illustrating an AR display.

FIG. 5 is an exemplary block diagram illustrating an AR displayincluding un-stocked virtual items.

FIG. 6 is an exemplary block diagram illustrating a customized AR filtercomponent.

FIG. 7 is an exemplary block diagram illustrating an AR overlay.

FIG. 8 is an exemplary block diagram illustrating a machine learningcomponent.

FIG. 9 is an exemplary block diagram illustrating a system 900 forupdating an AR display in real-time based on promotional offers.

FIG. 10 is an exemplary block diagram illustrating a user device forgenerating a real-time AR display.

FIG. 11 is an exemplary block diagram illustrating a set of sensordevices.

FIG. 12 is an exemplary block diagram illustrating a user devicescanning an identifier on an item 1202.

FIG. 13 is an exemplary flow chart illustrating operation of thecomputing device to generate a customized AR display for a user.

FIG. 14 is an exemplary flow chart illustrating operation of thecomputing device to generate a graphical representation of un-stockeditems in an AR display.

FIG. 15 is an exemplary flow chart illustrating operation of thecomputing device to update an AR display in real-time.

FIG. 16 is an exemplary flow chart illustrating operation of thecomputing device to apply customized item filters via AR.

Corresponding reference characters indicate corresponding partsthroughout the drawings.

DETAILED DESCRIPTION

Referring to the figures, examples of the disclosure customizedfiltering of items in an item selection area via augmented reality (AR).In some examples, a customized AR filter component generates an ARdisplay including a real-world image of a plurality of items overlaidwith graphical item indicators identifying items within a predetermineddistance of the user predicted to be of high-interest to the user and/oritems predicted to be of low-interest to the user. This enables the userto quickly and efficiently identify items which the user is interestedin selecting while greying out/hiding items the user is uninterested inviewing to save time and reduce/eliminate any frustration which canoccur due to viewing items of little/no interest to the user.

The customized AR filter component in other examples generates an ARdisplay of a portion of a real-world item selection area overlaid withone or more graphical elements representing un-stocked items predictedto be of interest to the user which are not physically present withinthe item selection area. The un-stocked items physically absent from theitem selection area can include items located in another part of a store(not within the user's immediate vicinity), items in a backroom or otherstorage area, items in transit to the store, items available for orderfrom a distribution center, items available for order from anotherstore/storage area, etc. This enables the user to select a desired itemfrom the selection of items currently available in the item selectionarea as well as items not physically present but available for order.This enables provision of a greater selection of items, increasedoptions available for users and increased likelihood that a user willlocate a desired item or substitute for an item the user is interestedin obtaining.

Referring again to FIG. 1, an exemplary block diagram illustrates asystem 100 for customized item filtering using augmented reality (AR).In the example of FIG. 1, the computing device 102 represents any deviceexecuting computer-executable instructions 104 (e.g., as applicationprograms, operating system functionality, or both) to implement theoperations and functionality associated with the computing device 102.The computing device 102 can include a mobile computing device or anyother portable device. In some examples, the mobile computing deviceincludes a mobile telephone, laptop, tablet, computing pad, netbook,gaming device, and/or portable media player. The computing device 102can also include less-portable devices such as servers, desktop personalcomputers, kiosks, or tabletop devices. Additionally, the computingdevice 102 can represent a group of processing units or other computingdevices.

In some examples, the computing device 102 has at least one processor106 and a memory 108. The computing device 102 in other examplesincludes a user interface component 110.

The processor 106 includes any quantity of processing units and isprogrammed to execute the computer-executable instructions 104. Thecomputer-executable instructions 104 can be performed by the processor106 or by multiple processors within the computing device 102 orperformed by a processor external to the computing device 102. In someexamples, the processor 106 is programmed to execute instructions suchas those illustrated in the figures (e.g., FIG. 13, FIG. 14, FIG. 15,and FIG. 16).

The computing device 102 further has one or more computer readable mediasuch as the memory 108. The memory 108 includes any quantity of mediaassociated with or accessible by the computing device 102. The memory108 can be internal to the computing device 102 (as shown in FIG. 1),external to the computing device (not shown), or both (not shown). Insome examples, the memory 108 includes read-only memory and/or memorywired into an analog computing device.

The memory 108 stores data, such as one or more applications. Theapplications, when executed by the processor 106, operate to performfunctionality on the computing device 102. The applications cancommunicate with counterpart applications or services such as webservices accessible via a network 112. In an example, the applicationsrepresent downloaded client-side applications that correspond toserver-side services executing in a cloud.

In other examples, the user interface component 110 includes a graphicscard for displaying data to the user and receiving data from the user.The user interface component 110 can also include computer-executableinstructions (e.g., a driver) for operating the graphics card. Further,the user interface component 110 can include a display (e.g., a touchscreen display or natural user interface) and/or computer-executableinstructions (e.g., a driver) for operating the display. The userinterface component 110 can also include one or more of the following toprovide data to the user or receive data from the user: speakers, asound card, a camera, a microphone, a vibration motor, one or moreaccelerometers, a BLUETOOTH® brand communication module, globalpositioning system (GPS) hardware, and a photoreceptive light sensor. Inan example, the user inputs commands or manipulate data by moving thecomputing device 102 in a particular way.

The network 112 is implemented by one or more physical networkcomponents, such as, but without limitation, routers, switches, networkinterface cards (NICs), and other network devices. The network 112 isany type of network for enabling communications with remote computingdevices, such as, but not limited to, a local area network (LAN), asubnet, a wide area network (WAN), a wireless (Wi-Fi) network, or anyother type of network. In this example, the network 112 is a WAN, suchas the Internet. However, in other examples, the network 112 is a localor private LAN.

In some examples, the system 100 optionally includes a communicationsinterface component 114. The communications interface component 114includes a network interface card and/or computer-executableinstructions (e.g., a driver) for operating the network interface card.Communication between the computing device 102 and other devices, suchas but not limited to, the user device 116 and/or one or more sensordevice(s) 118, can occur using any protocol or mechanism over any wiredor wireless connection. In some examples, the communications interfacecomponent 114 is operable with short range communication technologiessuch as by using near-field communication (NFC) tags.

The user device 116 represents any device executing computer-executableinstructions. The user device 116 can be implemented as a mobilecomputing device, such as, but not limited to, a wearable computingdevice, a mobile telephone, laptop, tablet, computing pad, netbook,gaming device, and/or any other portable device. The user device 116includes at least one processor and a memory. The user device 116 canalso include a user interface component. In this example, the userdevice 116 is an AR headset.

The set of sensor device(s) 118 is a set of one or more devices forgenerating sensor data 146 associated with the plurality of items 130.The set of sensor device(s) 118 can include, without limitation, one ormore image capture devices, microphones, radio frequency identifier(RFID) tag reader(s), barcode readers, robotic sensor devices, etc. Animage capture device can include a video camera and/or a still imagecamera for generating an image of at least one item in the plurality ofitems 130. A barcode reader can include a universal product code (UPC)reader, matrix barcode reader, quick response (QR) code reader, etc.

The system 100 optionally includes a data storage device 120 for storingdata, such as, but not limited to a set of user preferences 122 and/or aset of weighted selection criteria 124. The set of weighted selectioncriteria 124 includes one or more filtering rules for filteringhigh-interest items and low-interest items from the plurality of items.The weighted selection criteria 124 can include positive filtering rulesand negative filtering rules. Positive filtering rules are properties orcategories for high-interest items. The negative filtering rules includeproperties/categories for low-interest items.

The weighted selection criteria 124 in some examples include positive ornegative attributes for filtering items. A positive attribute is anattribute of an item that the user wants/desires. A negative attributefor filtering is an attribute of an item that is undesirable/unwanted bythe user.

The attributes can include ingredients, product dimensions,descriptions, price, supplier, manufacturer, type of packaging, aislelocation, UPC, origin, etc. Items can be filtered to select items madeusing recycled materials, item that are aluminum free, items containingno high-fructose corn syrup (HFCS), sugar-free, gluten free, or otherattribute.

In one example, the system 100 filters items to highlight/select itemsconforming to a vegan diet. In this example, positive attributes forfiltering include, without limitation, items having no animal products.The negative attributes for filtering can include, without limitation,animal products in the ingredients or composition of the item.

In other examples, a negative attribute for filtering (negative filter)includes an ingredient that is harmful to the user or unwanted by theuser, such as an allergen. For example, the user can have a negativefilter criterion for peanuts. In this example, all items having peanutsin the ingredients are filtered out. In another example, the selectioncriteria can indicate that only gluten-free items are selected, andgluten-containing items are filtered out.

In another example, the system identifies each instance of each type ofitem in accordance with a unique identifier, such as a UPC or RFID tag.Each instance of each type of item that is perishable can have its ownexpiration date. In these examples, the selection criteria includes anexpiration date or range of expiration dates. The system 100 in theseexamples filter items in accordance with a user selected expiration dateand/or a user selected range of expiration dates. This enables a user,such as a stocking manager, to identify items which are approaching anexpiration date for markdown, removal from shelves for disposal of theitem, and/or relocation of the item to a markdown area.

The weighted selection criteria 124 in another example specifies thenumber of filter results to be included in the final results, such asthe number of high-interest items to be identified. If the weightedselection criteria 124 specifies ten filter results, the final set ofhigh-interest items identified by the AR filter application will includeten or less items matching the selection criteria and/or theuser-provided search terms. In another example, if the weightedselection criteria 124 specifies twenty results, the AR display willinclude twenty or fewer items within the AR display having positive itemindicators. All other items not included in the twenty filter resultswill be obscured, greyed out, hidden or otherwise associated with anegative item indicator. This enables users to limit the search resultsto reduce resource usage on the user device.

The data storage device 120 can include one or more different types ofdata storage devices, such as, for example, one or more rotating disksdrives, one or more solid state drives (SSDs), and/or any other type ofdata storage device. The data storage device 120 in some non-limitingexamples includes a redundant array of independent disks (RAID) array.In other examples, the data storage device 120 includes a database.

The data storage device 120 in this example is included within thecomputing device 102 or associated with the computing device 102. Inother examples, the data storage device 120 is a remote data storageaccessed by the computing device via the network 112, such as a remotedata storage device, a data storage in a remote data center, or a cloudstorage.

The data storage device 120 in some non-limiting examples is utilized toaggregate data together for item filtering. The aggregated data caninclude item data, planogram data, aisle location data, promotionaldata, selection criteria, user preference data, transaction historydata, etc. This enables data utilized for item selection to beaggregated into a single location for quick and efficient access by theitem filter and/or the filter application on the user device.

In another example, the data storage device 120 stores inventory data,including inventory of items within the item selection area. In otherexamples, the data is aggregated on a cloud storage device rather than aphysical data storage associated with the item selection area.

The memory 108 in some examples stores one or more computer-executablecomponents. Exemplary components include a customized AR filtercomponent 126. The customized AR filter component 126, identifies alocation of the user device 116 associated with a user 128 and aplurality of items 130 within a predetermined distance of the userdevice 116 based on the identified location of the user device 116.

In some examples, the AR filter component 126 identifies the location ofthe user device 116 by analyzes sensor data generated by the user device116 and sent to the customized AR filter component 126 on the computingdevice 102 via the network 112. The sensor data received from the userdevice 116 can include images/video of items within a predeterminedrange of the user device 116, scanner data obtained by scanning items inthe plurality of items 130, and/or any other sensor data.

The customized AR filter component 126 identifies high-interest itemsand/or low-interest items from the plurality of items 130 by filteringitems in the plurality of items 130 within the predetermined range ofthe user. The predetermined range of the user in an example is the areaor range encompassing the FOV of the user device. An item ofhigh-interest is an item conforming with one or more filtercriteria/search terms or item attributes provided by the user. Alow-interest item is an item that does not conform/match desiredattributes/filter criteria provided by the user. The customized ARfilter component 126 receives the user-provided search terms/filtercriteria from the user device 116 via the network 112. In an example,the filter application includes security credentials enabling it toaccess the network 112. The user device 116 credentials can beauthorized via a user login, or other credentials.

The customized AR filter component 126 in some examples identifies oneor more un-stocked items of high-interest to the user. An un-stockeditem is an item that conforms to the user's desired attributes/searchcriteria but is not physically available within the plurality of items130. An un-stocked item can include items available in another part of astore, items available on request from a backroom or other storage area,and/or items available for order from another store, a distributioncenter or other remote location. This enables the item filtering systemto be linked to online/e-commerce item fulfillment resources.

The customized AR filter component 126 generates an AR overlay 132including one or more indictor(s) 134. The indictor(s) 134 includepositive item indicator(s) associated with each high-interest item inthe plurality of items 130, negative item indicator(s) associated witheach low-interest item in the plurality of items 130, and/or a graphicalrepresentation for each un-stocked item of potential interest to theuser 128.

An AR generator device 136 outputs an AR display 138 of a portion of anitem selection area within a field of view (FOV) 140 of the user 128,including the plurality of items 130. The AR display 138 is athree-dimensional image including real-world elements as well asvirtual/graphical elements generated by the AR generator device 136. TheAR display 138 includes the AR overlay 132 superimposed over thereal-world image of the items in the FOV of the user 128.

In other examples, the computing device 102 sends the AR overlay 132 tothe user device 116 via the network 112. A filter application 142generates the AR display 138. The AR overlay includes an identificationof filtered items (high-interest and low-interest items. An AR generator144 on the user device utilizes the AR overlay received from thecomputing device to generate the AR display 138 and output the ARdisplay 138 to the user 128. The user device 116 downloads the filterapplication 142 from a web applications server via the network 112.

The computing device 102 optionally performs the item filtering andgenerate filter data identifying high-interest items and/or low-interestitems for a selected user. The filter data, identifying low-interestand/or high-interest items, are sent to the user device 116 via thenetwork 112. The filter application 142 generates the AR display 138,including the AR overlay providing positive item indicators and negativeitem indicators based on the filter data received from the computingdevice 102. An AR generator 144 outputs the AR display 138 to the user128.

In still other examples, the customized AR filter component 126 isexecuted on the user device 116. In these examples, the customized ARfiler component identifies the location of the user device 116, filtersitems using selection criteria, generates the AR display 138 viathree-dimensional mapping, and outputs the AR display 138 to the user128 in real-time as the user views the plurality of items 130.

The AR display 138 is updated at predetermined time intervals in otherexamples. The AR display 138 in a non-limiting example, is updated everythree seconds. In another example, the AR display 138 is updated everyfive seconds. In other examples, the AR display 138 is updated inresponse to user input.

The user input in one example includes, without limitation, a verbalrequest for information associated with an item, a user gesturingtowards an item, a user touching or picking up a physical item withinthe FOV of the user device, a user placing a physical item in a physicalbasket/cart, a user touching or selecting an un-stocked virtual item ona virtual shelf/display within the AR display, a user selecting acontrol associated with an item within the AR display, a user changingan item arrangement or shelf/display arrangement within the AR display,and/or any other user input. In other examples, the user input includesuser behavior or movements, such as a hand gesture, pointing, toughingan item, removing an item from a shelf, clicking on an item, etc. Theuser input can include typing or speaking search terms or otherqueries/requests.

The user device 116 in some examples communicates with the computingdevice 102 or other local server on the Internet via web servicesapplication programming interface (API) management. The user device 116provides real-time AR capability to filter items based on customizedcriteria based on the user's preferences and prior transaction historyas the user walk's around a store shopping. The system 100 highlightsitems of interest to the user and/or blocks or obscures unwanted itemsbased on the filtering criteria and/or user-provided search terms oritem attributes desired by the user.

In one example, if the user associated with the user device 116 issearching for a skin care product but the user does not know exactlywhich skin care item would be most suitable for the user, the userdevice 116 filters the items in the plurality of items 130 to identifyone or more items having the desired attributes and matching userpreferences to facilitate location of an appropriate item in less timefor improved user satisfaction. The attributes/filter criteria enablethe user to narrow down the search. In the example above, the userindicates desired attributes such as, but not limited to, items designedfor use at night, moisturizer, non-comedogenic, cream or liquid, itemswithin a specified price-range, etc. The system 100 identifies itemsconforming to the specified attributes, preferences, etc.

In an example, the user specifies search/filter criteria such as, butnot limited to, fishing lures for catching big mouth bass. In thisexample, the system 100 identifies fishing lure items designed forcatching big mouth bass. Fishing lures which are not appropriate forcatching big mouth bass would be placed in the set of low-interest itemsand filtered out/associated with a negative item indicator forhiding/obscuring the items.

In another example, the user device 116 utilizes planogram data, aislelocation data, sensor data (image data), item data, and selectioncriteria to generate the AR display. The user device 116 obtains theplanogram data, aisle location data, item data, and/or other ARfiltering data from the computing device, a data storage device, orother data source via a request accompanied by the appropriate securitycredentials sent via the network 112. The request can include anidentification of a store (item selection area identifier) to ensure theplanogram, aisle location data, and item data is store-specific.

The computing device 102 optionally includes a set of sensor devices 145for generating sensor data 146. The set of sensor devices 145 includesone or more image capture devices, one or more scanner devices, arobotic scanner device, one or more RFID tag readers, or any other typeof sensor device. The set of sensor devices 145 generate sensor data 146such as, but not limited to, image data associated with one or moreitems in the plurality of items 130. The computing device 102 optionallyanalyzes the sensor data 146 using image recognition and item data todetermine a location of the user 128 and/or identify items in theplurality of items 130.

The computing device 102 performs the image analysis/image recognitionand item filtering. The user device 116 includes AR (overlay andinterpretation) and computer vision for understanding and contextmapping of the area/location awareness recognition. The user device 116generates the AR overlay and AR display utilizes the item filteringresults identifying high-interest items and low-interest items togenerate the AR display on the user device 116.

In other examples, the computing device 102 receives the sensor datafrom the user device 116 and generates the instructions for creating theAR display. The computing device 102 outputs the instructions to theuser device 116.

FIG. 2 is an exemplary block diagram illustrating a system 100 forper-user customized filtering of items via a user device, such as theuser device 202 or the user device 204. Sensor device(s) 206 generatessensor data associated with a plurality of items 130. The sensor datacan include image data (camera images) of the item, the set of shelves220, marker(s) 218 on the items, marker(s) 218 on the set of shelves, orany other identifying marks within the item selection area.

The plurality of items 130 includes items arranged on one or moreshelves in the set of shelves 220, such as, but not limited to, item210, item 212, item 214, and/or item 216. The plurality of items 130 caninclude one or more marker(s) 218 used to identify the items and/oridentify a location of the user device 202 and/or the user device 204.

The marker(s) 218 includes one or more markers associated with one ormore items in the plurality of items 130. The marker(s) 218 can includea UPC, matrix barcode, printed “text” label, symbol, RFID tag,watermark, graphic, a sign, a QR code, or any other mark for identifyingan item or a shelf in the set of shelves. A marker can be included on apaper label, sticker, paint, engraving, etc. The system analyzes sensordata associated with the marker(s) 218 with item data, planogram, and/oraisle location data to identify one or more items within a FOV or otherrange of the user or user device.

In one example, different clothing sizes are assigned a same UPC codebut have a different color tag. In a non-limiting example, large sizeclothing is given a red tag, medium size clothing is given an orangetag, and small size clothing has a green size. The system analyzes theUPC to identify the type of clothing and analyzes image data associatedwith the colored tags to determine a size of each instance of theclothing item. The system utilizes this data to filter clothing items inaccordance with a size of the user to enable the user to more quicklyand accurately identify clothing items in an appropriate size.

In another example, each instance of a garment has a different RFID tag.The system utilizes sensor data obtained from each RFID tag to identifygarments/clothing items in an appropriate size for the user. The correctsized clothing items are highlighted or otherwise associated with apositive item indicator. Clothing items in a different size than theuser's clothing size are filtered out.

The customized AR filter component analyzes sensor data generated by thesensor device(s) 206 to identify the items in the plurality of items 130and/or identify the location of the user device 202. In this example,the customized AR filter component executing on the user device 202generates a set of filtered items 222, including items 210 and 214 butexcluding items 212 and 216. The set of filtered items 222 includesitems filtered for the user 226. The set of filtered items 222 aredisplayed in an AR display 224 generated by the user device 202 withpositive item indicators. The AR display 224 is a three-dimensional ARimage including both real-world elements as well as virtual/graphicalelements. The AR display 224 is a display such as, but not limited to,the AR display 138 in FIG. 1.

The items excluded from the set of filtered items can be blocked,hidden, grayed out, obscured, or otherwise deleted/removed from the ARdisplay 224 to assist the user in identifying items of interest to thatuser. Thus, in some examples, the AR display hides or obscureslow-interest (filtered/invalid) items and highlights high-interestitems. The AR display can also provide additional information associatedwith high-interest items. Additional information is not provided forlow-interest items.

In an example, if the plurality of items 130 includes breakfast cereals,item 210 can include a brand “A” cereal, item 212 can be a brand “B”cereal, item 214 can be an organic variety cereal, and item 216 can be ageneric value cereal. If the user 226 prefers brand “A” items andorganic items, the customized AR filter includes item 210 brand “A” anditem 214 “organic” brand items in the set of filtered items for the user226.

The sensor device(s) 228 on a different user device 204 associated witha different user 232 viewing the same plurality of items 130 generatessensor data including images of the plurality of items 130. Thecustomized AR filter component on the user device 204 utilizes the userpreferences and/or transaction history data for the user 232 to generatea different set of filtered items 230 customized for user 232.

For example, if the user 232 prefers brand “A” items and lower priced,value items but has little/no interest in organic items, the set offiltered items can include item 210 brand “A” and item 216 value item.The set of filtered items 230 are presented within an AR display 234output to the user 232 via the user device 204.

In some example, the physical set of shelves 220 includes a digitaloutput device 235. The digital output device 235 can include, withoutlimitation, a light emitting diode (LED) display, a digital display, orany other type of digital output device. The digital output device 235outputs default content 236, including shelf identifiers, itemidentifiers, item pricing information, item size information,promotional information, as well as any other default content.

In other examples, the user device 202 sends customized content 238 tothe digital output device 235 for output to the user 226 when the userdevice detects the digital output device within a predeterminedrange/distance of the user device. In other examples, the digital outputdevice 235 displays customized content received from the user device forso long as the digital output device detects the user device within apredetermined range of the digital output device. A geofence area can beutilized to define the predetermined area. When the user device iswithin the geofence area, the digital output device displays thecustomized content received from the user device.

The customized content 238 can include content associated with itemspreferred by the user, such as high-interest items in the set offiltered items 222. In one example, the customized content includesinformation associated with item 210 and/or item 214.

When the user device 202 is no longer within the predetermined range ofthe digital output device 235, the digital output device 235 resumesdisplaying the default content 236. In other examples, when the userdevice 202 is detected within the predetermined range of the digitaloutput device 235, the user device 202 sends customized contentassociated with item 210 and/or item 216 to the digital output device235. The digital output device 235 outputs the customized content 238while the user deice 204 is within range of the digital output device235 for viewing by the user 232.

A geofence can be utilized to determine whether to display customizedcontent on the digital output device 235. In this example, if the userdevice is within a geofence area associated with the digital outputdevice 235, the digital output device 235 pings the user device torequest the customized content. In other examples, the user deviceautomatically sends the customized content associated with ahigh-interest item to the digital output device 235 in response todetecting/entering the geofence area. The digital output device 235displays the customized content as long as the user device is within thegeofence area. When the user device is no longer within the geofencearea, the digital output device 235 resumes display of default content.

The customized content 238 can be sent to the digital output device 235from the user device 202 via the network 112. The network 112 caninclude a BLUETOOTH®, a beacon transmitter, a LAN, a WAN, or any othertype of network.

FIG. 3 is an exemplary block diagram illustrating an item selection area300. The item selection area 300 is an area including item displaysand/or a plurality of items 130. The item selection area 300 can includea store or other retail environment. The item selection area 300 caninclude an indoor area and/or an outdoor area having one or more itemsdisplayed for selection and/or purchase by one or more users.

The plurality of items 130 includes any type of items. The plurality ofitems 130 can be displayed on the set of shelves 220. The set of shelves220 can include one or more shelves, a refrigerated display cabinet, afreezer display, a dairy case, a warm/heated display, a floor display,an end-cap display, a side counter, a gondola shelf unit, a modularaisle display, or any other item display area.

A user 306 associated with a user device 308 views an AR displayincluding a real-world image of a portion of the item selection area 300within a FOV 310 of the user 306 or the FOV of the user device 308overlaid with an AR overlay. The user device 308 is a computing device,such as, but not limited to, the computing device 102 in FIG. 1, theuser device 116 in FIG. 1, the user device 202 in FIG. 2, and/or theuser device 204 in FIG. 2. In this non-limiting example, the user device308 is an augmented reality glasses or head-set. In other examples, theuser device 308 includes a tablet, cellular telephone, or other mobilecomputing device.

In some examples, the item selection area 300 includes one or moresensor devices for identifying a location of the user device within theitem selection area 300. For example, the item selection area 300includes image capture devices, beacon transmitters, beacon receivers,infrared (heat) sensors, proximity sensors, etc. The system in theseexamples analyzes the sensor data generated by the sensor device(s) todetermine when an identified user is within proximity to a digitaloutput device or other display area for customizing displayed content.In an example, IR sensor data is utilized for three-dimensional mappingof an area associated with the user device to identify a location of theuser device within the item selection area and/or identify a pluralityof items located within a given range of the user device.

FIG. 4 is an exemplary block diagram illustrating an AR display 138. TheAR display 138 is a three-dimensional image including real-world imagesof a portion of the item selection area, including a set of filtereditems 402 physically present on a set of shelves 220. The set of shelves220 is a set of real-world shelves physically present in the itemselection area. The set of filtered items 402 includes items havingpositive item indicators and/or items having negative item indicators.Positive item indicators can include highlighting, bolding, color,flashing indicators, enlarged graphical image of the item, arrows orother indicators identifying one or more items as items predicted to beof interest to the user. The negative item indicators can includegreying out, covering/hiding, removing the item, etc.

In some examples, the set of shelves 220 in the AR include high-interestitems physically present on the shelf as well as one or more un-stockedvirtual items not physically present on the shelf which is representedby a graphical element, such as item 405. In this example, the graphicalelement 405 mimics an appearance of a three-dimensional item. Forexample, if the set of high-interest items includes olive oil andcoconut oil jars which are physically present on the shelf and thesystem determines the user can be interested in avocado oil, which iscurrently out-of-stock on this shelf, but available in thebackroom/storage, the system can add a three-dimensional graphicalelement representing the jar of avocado oil displayed on the set ofshelves 220. If the user selects the jar of avocado oil, the systemsends a notification to another user, such as a stocking manager, tore-stock the item and/or redirects the user to an order form/webpagewhere the user can order the item for deliver or later pickup.

The set of un-stocked virtual items 406 is a set of one or more itemsnot physically present within the FOV 310 of the user 306 and/or theuser device 308. An un-stocked item can include an item available inanother location within the item selection area (different shelf),available in a storeroom/backroom, available for order from adistribution center, available for order from another store, etc. Thegraphical representation of each un-stocked item in the set ofun-stocked virtual items 406 can be displayed on one or more virtualshelves or other displays.

The set of virtual shelves 408 includes a graphical representation ofone or more shelves or other displays that are not physically present inthe item selection area. The set of virtual shelves 408 can be referredto as “ghost” shelves. A virtual shelf or other virtual item display canbe displayed within the AR display 138 overlaid over areas of the itemselection area 300 where a real-world physical shelf/display could notbe located. In a non-limiting example, the AR display 138 includes avirtual shelf/display overlaid on the real-world location of a wall, awalkway/aisle, in front of a door/elevator, or other location wherephysical shelves could not be placed. In another example, a graphicalrepresentation of the virtual shelf in the set of virtual shelves 408hovers off the ground or floating above a physical shelf within the ARdisplay.

FIG. 5 is an exemplary block diagram illustrating an AR display 138including un-stocked virtual items. The AR display 138 in this exampleincludes physical display(s) 502 including filtered physical items 504and/or graphical representation of un-stocked item(s) 506. The physicaldisplay(s) 502 include an image of a physical display. The filteredphysical items 504 includes images of physical items within the itemdisplay area conforming to the user's preferences/filter criteria. Inother words, the filtered physical items 504 is an item physicallypresent in the item selection area that is predicted to be of interestto the user based on the user's preferences, the user's transactionhistory data, and/or the selection criteria.

The virtual display(s) 508 is a graphical representation of one or moredisplays within the AR display which are not physically present in theitem selection area. The virtual display(s) 508 includes un-stockedvirtual items(s) 512 available from another location for request, order,deliver, and/or later pickup.

The AR display 138 can include status indicator(s) 514 associated witheach item in the filtered physical items 504 and/or the un-stockedvirtual item(s) 512. The status indicator(s) 514 can include analternate location indicator 516 indicating that an un-stocked item isavailable at another location within the item selection area. A re-stockindicator 518 associated with an item in the AR display indicates anitem is available in a backroom or other storage area in the itemselection area. For example, if the user's preferences, search terms(desired item attributes), and/or other filter criteria indicate an itemcurrently out-of-stock (OOS) on a shelf but available in abackroom/storage area, the system provides a graphical representation ofthe item which is not physically present on shelf with a re-stockindicator. If the user selects the re-stock indicator, the system sendsthe re-stock notification to a restock manager requesting additionalinstances of the OOS item from the backroom/storage area be brought tothe sales floor for restocking the shelf and/or bringing a requestednumber of instances of the OOS item to the user requesting the restock.

An order indicator 520 indicates an un-stocked item is available fororder from another remote location, such as, but not limited to, adistribution center or another store/local marketplace, for delivery tothe user and/or later pickup at the item selection area.

Control(s) 522 can be provided, such as, but not limited to, restockcontrol 524. User selection of the restock control 524 triggers sendinga notification to another user to restock the physical display and/orbring one or more items from the backroom/storage area to the user. Thisenables the item filtering system to be linked to inventory forre-ordering/restocking items and/or updating inventory. The restockingoption is provided if sufficient numbers of items are available withinthe on-hand inventory.

The item order request control 526 when selected by a user triggerssending a link or order form to the user device associated with theuser. The location request control 528 can be selected by the user toobtain map 530 and/or directions 532 to another area or other locationwithin the item selection area where a desired item can be found. Forexample, if the filtered physical items 504 includes black printer ink,the virtual display can include a set of virtual items associated withprinter ink, such as, but not limited to, printer paper, frequently usedwith printer ink but not physically present on the same shelf as theprinter ink. The user can select the location request control 528 toobtain map and/or directions to an aisle or other area of the itemselection area where printer paper is displayed. The map and/ordirections can be updated in real-time as the user is moving/walkingthrough the item selection area. If a user turns down the wrong aisle,the map and/or directions can be updated in real-time to redirect theuser to the correct location. The directions can include arrows and/ordirection indicators within the AR display directing the user to thelocation of another item.

The AR display 138 optionally include a streaming content display node534 providing streaming content in real-time. The streaming contentdisplay node 534 is a graphical node, window, or other graphical elementwithin the AR display 138 providing streaming content customized and/orselected by the user as the user views items in the item display area.The steaming content can include videos, movies, television shows,commercials, links to manufacture/supplier website, item reviews, itemdemonstration videos, etc.

The AR display 138 in other examples includes a set of one or morevirtual sign(s) 538, including information 540 associated with one ormore items predicted to be of interest to the user. A virtual sign is asign which is not a physical sign in the real-world. The virtual signexists only as a three-dimensional, graphical element within the ARdisplay 138. The information 540 provided within a virtual sign caninclude, without limitation, name of items, pricing information, sizeinformation, ingredients, promotions/discounts, rebates, coupons, or anyother information associated with the items.

FIG. 6 is an exemplary block diagram illustrating a customized AR filtercomponent 126. The customized AR filter component 126 is a componentrunning on a computing device for filtering items in real-time via an ARdisplay. The customized AR filter component 126 includes an item filtercomponent 602 in this example.

The item filter component 602 analyzes item data 604 associated with oneor more items within a FOV of a user device in the item selection area.The item data 604 can include ingredients, brand name, nation of origin,state of origin, seasonality, price, coupons, discounts, rebates,promotions, organic certification, sugar content, fat content, glutencontent, etc.

The item filter component 602 analyzes the item data 604 in someexamples using a set of weighted selection criteria and a set of userpreferences. The item filter component 602 identifies a set ofhigh-interest items 606 and a set of low-interest items 612 customizedfor a selected user based on the analysis.

The set of high-interest items 606 can include items in the same orsimilar brand-line as a selected item. Items in the same brand-line areitems designed to be used together or in conjunction, such as, but notlimited to, a bottle of shampoo and a bottle of conditioner by the samebrand. Thus, if a user selects a bottle of shampoo in brand “A”, the setof high-interest items 606 can include brand-line items for brand “A”,such as brand “A” condition, hair spray, anti-frizz cream, etc.

The set of high-interest items 606 can include items frequently usedtogether. For example, if the user selects chips, the set ofhigh-interest items 606 can include salsa or guacamole dip. In anotherexample, if a user selects a box of cereal, the set of high-interestitems 606 can include milk.

The set of high-interest items 606 can also reflect purchase patterns bya selected user. For example, if the user frequently purchases organicitems, the set of high-interest items can include all organic varietiesof items within the FOV of the user or the FOV of the user deviceassociated with the user. In another example, if the user demonstratesbrand-loyalty for a brand “B”, the set of high-interest items 606 caninclude all items associated with brand “B” within the FOV of the useror the FOV of the user device associated with the user.

The set of high-interest items 606 is a set of one or more itemsphysically present 608 within the FOV of the user device. In otherexamples, the set of high-interest items can also include a set of oneor more items physically absent 610 from the FOV of the user devicepredicted to be of interest to the user. The high-interest items absentfrom the FOV of the user can be referred to as un-stocked items.

The set of high-interest items 606 can include sub-sets of items havingvarying degrees of compliance with search terms. For example, the set ofhigh-interest items can include a first sub-set of highest/best matchingitems which have attributes matching most of the user's indicatedpreferences/filter criteria ninety percent match to one-hundred percentmatch). A second sub-set of items can include items having attributesconforming to fifty to seventy-five percent of the user's desiredattributes, and a third sub-set of items can include items matchingone-third to one-half of the user's desired attributes, and so forth.

In another example, the set of high-interest items 606 includes a firstsub-set of items the user has previously selected/purchased, a secondsub-set of items that are new items recently added to inventory whichthe user has not tried/may not be aware of, and/or a third sub-set ofitems having promotional offers associated with them.

An overlay component 614 assigns a positive item indicator 616 to eachitem in the set of high-interest items, such as, but not limited to, theitem 618. The positive item indicator 616 can include a graphicalelement highlighting the item 618, enlarging an image of the item 616, aflashing indicator, a strobing indicator, a color overlay, etc. In oneexample, the overlay component 614 assigns a first positive indicator topreviously purchased items which the user has prior experienceutilizing, a second positive item indicator to high-interest items whichare new/the user has never tried, and/or a third positive item indicatorassociated with promotional high-interest items which are associatedwith a bonus/larger package size, rebate, coupon, BYGO offer, reducedprice, or another discount. In this example, the first positive itemindicator, the second positive item indicator and the third positiveitem indicator each have a different appearance. In other words, thefirst positive item indicator looks different than the second positiveitem indicator and the third positive item indicator.

In still other examples, the overlay component 614 assigns a differentpositive item indicator for items in the set of high-interest items thatconform the closest to the user's preferences and/or search filters,such as items having seventy-five to one-hundred percent match withfilter/search terms. Another different positive item indicator to itemsin the set of high-interest items having less than the seventy-fivepercent conformance/matching with the user's search/filter terms. Inanother example, a different third positive item indicator can beassigned to items having fifty percent or less matching with the user'ssearch criteria/filter terms. In this manner, high-interest items aredistinguished based on which items are a best match with the desiredattributes and which high-interest items have least/fewest of thedesired attributes.

The overlay component 614 assigns a negative item indicator 620 to eachitem in the set of low-interest items, such as the item 622. Thenegative item indicator 620 can include greying out the item, shrinkingor minimizing an image of the item, removing an image of theitem/covering or hiding an image of the item, etc.

The overlay component 614 in other examples can include an item statusupdate 624. The item status update 624 is an update changing a negativeitem indicator to a positive item indicator in response to receipt of anew promotional offer 626 associated with the item and/or receipt of newuser input indicating interest in the item, type of item, and/orclassification of an item.

For example, if the user is only interested in items within a givenprice range and a new promotional offer 626 is received in real-timewhich lowers the price of the item 622 bringing it within the pricerange, the item can be moved from the set of low-interest items to theset of high-interest items. In this example, the negative item indicatorfor the item 622 can be changed to a positive item indicator reflectingthe lower price in response to the promotional offer 626.

The promotional offer 626 can be received by a promotions component 628in some examples. The promotions component 628 is a component forreceiving promotional offers in real-time from one or more suppliers viaa network connection, such as the network 112 in FIG. 1 and/or FIG. 2.The promotions component 628 analyzes promotional offers in real-timeusing the selection criteria and user preferences to determine whetheran offered promotion qualifies a low-interest item to be moved to theset of high-interest items. This enables the filter to utilize real-timebids or supplier-provided coupons to reduce cost or increase potentialuser interest in an item previously filtered-out/identified as alow-interest item.

In another example, if the user had previously expressed no interest inorganic items, but while shopping the user verbally requests informationon organic options, the item filter component can move all organic itemsfrom the set of low-interest items to the set of high-interest items inresponse to the user input indicating a new interest/preference fororganic items.

The customized AR filter component 126 optionally include a pop-upgenerator 630. The pop-up generator 630 outputs additional information632 associated with at least one item in the set of high-interest items606 within the AR display in response to receiving a user input 634associated with the at least one item.

The user input 634 can be received via a user interface, such as, butnot limited to, the user interface component 110 in FIG. 1. Theadditional information 632 includes item data, online purchaseinformation, link(s) to website for ordering/requesting an item, arestocking request control, suggested recipes, suggested brand-lineitems to be used in conjunction with a selected item, suggested relateditems to be used in conjunction with a selected item, etc.

The user input includes input provided by the user, such as verbalinput, gestures, selection of an item, text input, etc. For example, thesystem utilizes analytics to determine which items the user picks up,looks at an item, touches, puts back on the shelf, places in a cart, howlong the user looked at an item, etc. The verbal input can includesearch/filter terms, such as, but not limited to, desired itemattributes and/or undesired item attributes, etc. If a user looks at anitem for five seconds, it can be more significant/indicate greaterinterest than if the user only looks at it for one second.

A feedback component 636 in some examples outputs a request 638 forfeedback 640 associated with an item arrangement 642 within the ARdisplay or feedback associated with one or more item(s) 644 within theAR display selected for purchase by the user. Item arrangement 642feedback can be generated by user(s) rearranging items and/or virtualdisplays in the AR display. For example, if the AR display shows agraphical representation of children's cereals on a top shelf and highfiber/low sugar cereals on a lower virtual shelf, the user can rearrangethe AR display via user input to move the children's cereals to a lowershelf and move the lower sugar cereals to a higher shelf

In some examples, feedback 640 associated with item arrangement 642aggregated from a plurality of users can be analyzed to identifypreferred item placement/preferred item arrangements. This feedback canbe utilized to redesign and/or update physical item displays within theitem selection area.

The item(s) 644 feedback 640 can include feedback associated withwhether items in the set of high-interest items 606 are selected by theuser, whether the user indicates a preference for items placed in thelow-interest set of items, feedback rating items, etc.

A notification component 646 is a component for sending a notificationto one or more users. The notification component 646 can send arestock/out-of-stock notification to a first user/restock manager torestock a shelf in response to a second user selecting a restock controlwithin the AR display requesting restocking of an item. The notificationcomponent 646 in other examples sends a notification to a first userrequesting assistance for a second user in locating an item orretrieving an item that is currently on a shelf too high for the seconduser to reach.

The AR display generated by the customized AR filter component 126 isupdated in real-time on a regular interval, such as, but not limited to,a predetermined time interval 650. The predetermined time interval canbe any user-defined time-period at which new image data or other sensordata is generated for updating the AR display. The predetermined timeinterval 650 can be, without limitation, a one second time interval, afive second time interval, a thirty-second-time interval, or any othersuitable time-period for updating an AR image.

The customized AR filter component 126 performs filtering on itemswithin a predetermined distance 652 of a user device associated with theuser in some examples. The predetermined distance 652 is a range or zoneassociated with the user device. The predetermine distance 652 caninclude the FOV of an image capture device associated with the userdevice. The predetermined distance can be a user-defined radius utilizedto define an item filtering zone, such as, but not limited to, five feetradius, eight feet radius, or any other measurement.

FIG. 7 is an exemplary block diagram illustrating the AR overlay 132.The AR overlay 132 includes a set of positive item indicators 702, a setof negative item indicators 704 and/or a set of un-stocked virtual items706. The set of un-stocked virtual items 706 can include a graphicalrepresentation 710 of one or more physically absent item(s) 708.

FIG. 8 is an exemplary block diagram illustrating a machine learningcomponent 800. The machine learning component 800 analyzes per-userselection criteria 802 using feedback 804, training data 806, userpreferences 808, and/or historical transaction data 810.

The feedback 804 can include item review feedback, feedback associatedwith arrangement of items on a display, and/or feedback associated withaccuracy of customized item filtering. If the system filters items toidentify high-interest items the user does prefer and filters items theuser is uninterested in viewing, the feedback can be good/indicatingaccurate filtering. If the system selects high-interest items which arenot actually of interest to the user or if desired items are filteredout, the feedback can be poor.

The user preferences 808 can include user-selected AR displaypreferences. For example, the user preferences 808 can include auser-selected color for display of virtual shelves within the ARdisplay. In another example, the user preferences can specify apreferred item arrangement, such as child-related items on lower shelvesor candy/sweets on higher shelves within the AR display.

The machine learning component 800 utilizes real-time data, such as thefeedback 804 to adjust weights associated with each of the selectioncriterion. For example, if selection criteria indicate the user prefersto save money/emphasize value but the user also prefers brand-nameproducts, the selection criteria are weighted to indicate whichcriterion has greatest precedence.

The machine learning component in some examples utilizes the user'sfeedback and transaction data associated with items the user purchasesto adjust the selection criteria weights. In the example above, if theuser frequently purchases brand name items using coupons or inconjunction with other promotions, the machine learning component 800can generate weighted selection criteria 812 indicating greatestpreference should be given to promotions/discounts associated with brandname items.

FIG. 9 is an exemplary block diagram illustrating a system 900 forupdating an AR display in real-time based on promotional offers. Acomputing device 102 running a customized AR filter component 126 canreceive a promotional item 906 associated with a low-interest item 908from a remote computing device 910 associated with a supplier of thelow-interest item 908. The remote computing device 910 in some examplesreceives a notification from the customized AR filter component 126identifying an item provided by the supplier 912 as a filtered(low-interest) item. The supplier 912 can dynamically send thepromotional offer 906 in real-time to the customized AR filter component126 via the network 112. The supplier receives a real-time alertregarding filtering of the item. The supplier has an opportunity to bidand/or offer discounts or coupons in real-time to change filtering ofitems provided by the supplier. The promotional offer can include aprice discount, offer of a free item with purchase of an item, a rebateoffer, etc.

In another example, the promotional offer can include a supplier feepaid by the supplier to move an item provided by the supplier from theset of low-interest items into a set of neutral (unfiltered items)and/or into the set of high-interest items. A neutral/unfiltered item isan item which is displayed within the AR as it would normally appear inthe real-world without any alterations. In other words, a neutral itemwould be displayed without positive item indicators, such ashighlighting. Neutral items are also displayed without negative itemindicators, such as greying out, obscuring, etc.

In still another example, a supplier can provide a fee or otherpromotional offer to the user and/or to the system to qualify an itemfor placement within a virtual shelf/ghost shelf of items which do notmatch search/filter terms, but which are displayed within the AR displayfor viewing without negative item indicators. In some examples, if auser indicates a desire for items within a given price range, the systemcan display items which are outside the price range but close (within apredetermined upper end of the price range/exceeding maximum price by apredetermined amount) on a separate virtual shelf for viewing by theuser if the supplier of those items provides a promotional offer orother incentive associated with the item.

In one example, if a user specifies a maximum price of five dollars foran item, the system can include items having a price between fivedollars and seven dollars on a virtual shelf without negative itemindicators. In another example, items having a price exceeding thefive-dollar maximum but associated with a rebate or other promotion,such as a buy-on-get-one (BYGO) free offer can be displayed on aseparate virtual shelf with a promotional discount indicator informingthe user that these items are within the specified price range after thepromotion.

The customized AR filter component 126 can generate an updated ARdisplay 916 in response to the promotional offer 906. The updated ARdisplay 916 changes the low-interest item to a high-interest item.

FIG. 10 is an exemplary block diagram illustrating a user device 1002for generating a real-time AR display. A filer application 1103 on theuser device 1002 aggregates data from a plurality of sources inreal-time and utilizes the aggregated data to generate and/or update theAR display dynamically. The filter application is an application forfiltering items in an AR display, such as the filter application 142 inFIG. 1. In some examples, the user device obtains data from a datastorage, such as the data storage device 120 in FIG. 1 or a cloudstorage.

The user device 1002 receives promotional offers/promotional data from asupplier incentive database 1004. The supplier incentive database 1004includes data associated with rebates, coupons, discounts, sales, orother promotions associated with one or more items. In some examples,the promotions include discounts or incentives associated with new itemsto encourage a user to try a new item. This enables suppliers to get newitems added onto a user's filtered item list and/or get an item that hasbeen filtered out added back onto the high-interest items list orneutral items list. A neutral item is an item that is not in thehigh-interest item list or the low-interest filtered item list. Neutralitems are not associated with positive item indicators or negative itemindicators.

For example, if a user's preferences specify filtering items outside auser-selected price range, the supplier can offer a promotion, such as acoupon or other discount, to lower a price of a given item down into theuser's selected price range. This enables a supplier to identify an itemwhich would be filtered out due to price and offer to discount the itemto prevent the item from being filtered out due to price in real-time.The system dynamically adjusts filtering of items based on real-timedata received from the user searching for an item as well as suppliersof items.

Planogram 1006 data includes data describing where items arelocated/placed on shelves or other displays. The planogram 1006 can alsoinclude information, such as, the number of each type of item on eachshelf/display, schematics, visual representations of items in store,diagram/model indicating assigned location(s) of items onshelves/displays, and/or store layout.

The aisle location data 1008 identifies what types of items are on eachaisle, what items are on each shelf on each aisle, orientation ofshelves of aisles, number of items on each aisle, etc. The item filtercomponent cross-references planogram 1006 and aisle location data withitem image data to identify the location of the user device. The systemin some examples sends planogram data, aisle location data, and itemdata for items within the identified location of the user device to theuser device for utilization by the user device in creating the ARdisplay.

Sensor data 146 is data obtained from one or more sensor devices, suchas the sensor devices 118 in FIG. 1, the set of sensor devices 145 inFIG. 1, the sensor device(s) 206 in FIG. 2, and/or the sensor device(s)228 in FIG. 2. The sensor data 146 can include image data of a portionof the item selection area including one or more items.

Transaction history data 1012 includes a history of items selected by auser during one or more previous transactions. The transaction historydata 1012 can include an identification of high-interest itemspreviously selected by the user and/or items identified as low-interestitems which the user did select. This transaction data can be used tofine-tune the selection criteria and improve identification of itemslikely to be of interest to the user and items of low-interest or nointerest to the user.

Filter qualities per category 1014 include one or more item selectioncriteria for identifying categories of items of interest to a user. Thefilter qualities per category can include categories such as, low-sugarcategory of items, gluten-free category of items, organic category ofitems, value category of items, family size category of items,baby-related category of items, pet-related category of items, and othercategories.

The filter qualities per category 1014 can also include the number ofcriteria applied and/or the number of filter results output to the user.In an example, the filter qualities per category 1014 indicates a limitof five selection criteria to be applied during item filtering. Inanother example, the filter searches for first criteria limiting itemsto perfume, second criteria of women's perfume, third criteria of abrand of perform, fourth criteria of a price range, and fifth criteriaindicating a preferred size (sample size, one-ounce bottle, etc.). Oncethese five filters are provided by the user, the filtering begins toobtain a first round of filtering results. If the user wants to addadditional criteria, they can be added during a second round offiltering to fine-tune or further narrow the search results. In theseexamples, the output AR display only contains the user-specified numberof items/filter results. This reduces the processor resources consumedduring the AR filter to be limited/controlled.

For example, if the user is interested in the category of skin careitems, the filters identify moisturizers, cleansers, sunscreen, or otheritems in the skin care category of potential interest to the user.Likewise, if the user indicates a desire to locate skin-care items foruse at night, the filters identify night creams, makeup removers, andother skin care items designed for utilization at night. The filtersnarrow down the available items to assist the user in locating a desireditem more quickly and efficiently while enabling the user to view theitems in three-dimensional AR displays.

The item data 1016 includes data describing items. The item data 1016can include ingredients, origin of item, country of manufacture,seasonality, price, size, number of items per unit, etc.

Image database 1018 is a database of image data, including, withoutlimitation, item dimensions, packaging images, colors, itemrelationships, etc. The image database 1018 can also include image dataassociated with shelves, displays, and/or items in the item selectionarea. For example, the image database 1018 can include image dataassociated with a bottle of ketchup. The item filter component performsimage recognition to identify items on the shelves/displays within theFOV of the user/user device, identify an empty shelf due to out-of-stockitems, identify a location of the user device, etc.

The image recognition analysis is performed to compare image datagenerated by a user device with the known appearance of items/shelves toidentify one or more items. In one example, if the system recognizesfour different items together in one location, the system compares thoseitems with planogram and aisle location data to determine thethree-dimensional location of those four items within the item selectionarea.

For example, the system compares a known image of a bottle of ketchup,ketchup bottle label, ketchup bottle markers, or other identifying imagedata associated with ketchup bottles in inventory with image datagenerated by sensor devices associated with the user to identify bottlesof ketchup in the FOV of the user device. This image data can beutilized to identify items within a FOV of the user/user device and/oridentify the location of the user/user device. The known image data caninclude item dimensions, package color, label, bottle shape, bottlesize, box size, label size, label design, etc.

In some examples, the system performs image analysis based onidentification/comparison of markers on items where the items are verysmall in size and/or where item appearance or packaging for differentitems is very similar. In these examples, a marker on each item, such asa barcode, UPC, label, text on a label, image or graphic on a label,RFID tag, black and white marker, or other identifying marking on anitem or shelf associated with an item is utilized to identify itemswithin range of the user for AR item filtering. For example, if the useris looking at very small bottles of perfume or makeup, where all thepackage sizes are very similar, the filter component analyzes images ofmarkers on the packages to identify which item the user is viewingrather than analyzing package size, color, or item appearance. Whenmarkers/items are identified, that information can be mapped to aplanogram and/or aisle information data to determine a location of theuser and/or user device, as well as identify items within apredetermined range of the user.

The analytics database 1020 includes algorithms/analytics used toperform item filtering within the AR. The analytics can include an itemfilter 1024 for identifying high-interest items and/or low-interestitems. The analytics can also include machine learning (ML) 1022 forfine-tuning item selection criteria. The machine learning 1022 caninclude pattern recognition and artificial intelligence (AI).

In another example, the analytics database 1020 includes item identifierdata, such as UPC data, RFID data, barcode data, matrix barcodeidentifiers, etc. The analytics database 1020 can also include a storenumber, timestamp associated with the item filtering, productcategories, selected filters (positive and negative filters), and/orshopping instance for grouping date. A positive filter includesdesirable items, categories, types, or properties of items. If a userprefers a specific brand, that brand can be used as a positive filter. Anegative filter includes undesirable items, categories, types, orproperties of items. For example, if a user is only interested ingluten-free items, gluten is a negative filter and gluten-free can beused as a positive filter.

The analytics database 1020 can provide data feeds into marketingtrends. In other words, the analytics database 1020 aggregates datawhich can be analyzed to identify trends and local item preferences atan item-store level. The analytics database 1020 can also provide datafeeds into a supplier incentive database 1004. The supplier incentivedatabase 1004 can use filter data to display specific incentives forcustomer. The supplier incentive database 1004 allows the item filter toun-filter items that have incentives.

The analytics data is utilized to generate the AR display with thecustomized item filtering. In some examples, the system sends updatedanalytics data reflecting changes in the FOV of the user device, changesin items within the FOV, user input and/or new promotional data, at apredetermined time interval (every “x” seconds).

In these non-limiting example, the user device 1002 is a computingdevice such as the user device 116 in FIG. 1. The user device 1002aggregates data from a plurality of sources for utilization during theitem filtering, such as the supplier incentive database 1004. However,in other examples, the data is aggregated from the plurality of sourcesby a different computing device, such as the computing device 102 inFIG. 1. In still other examples, the data from the plurality of sourcesis aggregated onto a data storage for retrieval by the user device 1002.

FIG. 11 is an exemplary block diagram illustrating a set of sensordevices 145. The set of sensor devices 145 includes one or more imagecapture device(s) 1102 for generating a real-world image 1104 of aportion of an item selection area 1106. The set of sensor devices 145can also include microphone(s) 1108 generating audio data 1110, such as,but not limited to, verbal queries/requests for assistance, verbalrequests for information, and/or entering verbal search terms.

The set of sensor devices 145 optionally include RFID tag readers 1112for reading RFID tags associated with one or more items. The RFID tagreaders 1112 generate RFID tag data 1114. The RFID tag data 1114 can beutilized to identify the location of items and/or the location of theuser device.

Scanner(s) 1116 include UPC readers, matrix barcode readers, QR codereaders, etc. The scanner(s) 1116 generate scan data 1118 utilized toidentify item(s) within the FOV of the user and/or the user device.

Global positioning system (GPS) 1120 devices can be utilized to identifythe location of an item, a display, a scanner, and/or a user device. TheGPS data can be utilized to generate geofence data 1122. The geofencedata can be utilized to determine when a user device is within range ofa digital output device, such as, but not limited to, the digital outputdevice 235 in FIG. 2. When the user device is within the geofence areaassociated with a digital output device, the digital output devicedisplays content received from the user device customized for the user.When the user device is outside the geofence area, the digital outputdevice displays default content.

In other examples, the set of sensor devices include light sensors,proximity sensors, weight sensors, beacon receivers/transmitters, or anyother types of sensor devices. The sensor data can be analyzed by theitem filter component to identify items and/or identify a location ofthe user/user device.

FIG. 12 is an exemplary block diagram illustrating a user devicescanning an identifier on an item 1202. The item 1202 is an itemphysically available on a physical shelf or other physical displaywithin the item selection area. The item 1202 includes a marker 1204,such as a UPC, an RFID tag, a label, a black and white marker, oranother identifier. The user device 1206 includes a sensor device, suchas a scanner 1208, for reading the marker 1204. The user device 1206utilizes the scan data generated by scanning the marker to identify theitem 1202. In some examples, the user device 1206 utilizes a planogramand/or aisle location data to identify a location of the user devicebased on the identification of the item(s) and/or shelves(s) within theFOV of the user device.

In one example, the user device scans markers on a set of items withinthe FOV of the user to identify pickles, ketchup, mustard and othercondiments. The user device analyzes planogram and aisle location datato identify the aisle, shelf, and/or other location identifiersassociated with the assigned planogram locations of the pickles,ketchup, mustard and other condiments. The user device utilizes thisinformation to determine the current location of the user device and/oruser associated with the user device.

In other examples, the item filter identifies items by analyzes imagedata including images of the item(s). The item filter compares the imagedata with image data of one or more images in an image database, such asthe image database 1018 in FIG. 10.

In other words, if the user device determines the identification ofitems within proximity to the user device, the user device locates thoseitems on the planogram and aisle location data to determine the locationof the user device within the item selection area.

The planogram data, aisle location data, image data, filter qualities,and item data are stored (aggregated) together in a single database insome examples. This enables the user device to obtain the planogramdata, aisle location data and item data for utilization in generatingthe AR display using a single query to the database.

In another example, the planogram data and aisle location data is bestored in one database while the item data is stored in a separatedatabase. The user device sends a query to an orchestration application.The orchestration application aggregates the planogram data, aislelocation data, and item data from multiple sources and sends the data tothe user device.

FIG. 13 is an exemplary flow chart illustrating operation of thecomputing device to generate a customized AR display for a user. Theprocess shown in FIG. 13 can be performed by a customized AR filtercomponent 126, executing on a computing device, such as the computingdevice 102 or the user device 116 in FIG. 1.

The process begins by analyzing item data associated with a plurality ofitems within a FOV of a user device using weighted selection criteriaand user preferences at 1302. The plurality of items includes itemsphysically present within at least one display in an item selectionarea, such as, but not limited to, the plurality of items 130 in FIG. 1,FIG. 2, and/or FIG. 3.

An item filter component determines whether there are any high-interestitems at 1304. A high-interest item is an item predicted to be ofinterest to the user based on the user's preferences, the user'stransaction history, and/or a set of selection criteria. A high-interestitem can be an item such as items in the set of high-interest items 606in FIG. 6. The item filter component is a customized item filter, suchas the item filter component 602 in FIG. 6.

If the plurality of items does include any high-interest items, the itemfilter component assigns a positive item indicator to each high interestitem at 1306. A positive item indictor is a graphical element indicatinga high-interest item, such as, but not limited to, the positive itemindicator 616 in FIG. 6 and the set of positive item indicators 702 inFIG. 7.

The item filter component determines whether the plurality of itemsincludes any low-interest items at 1308. If yes, the item filtercomponent assigns a negative item indicator to each low interest item at1310. The negative item indicator is a graphical element associated withan item in the AR display indicating a low-interest item, such as, butnot limited to, the negative item indicator 620 in FIG. 6 and/or the setof negative item indicators 704 in FIG. 7.

An AR generator generates an AR display including the positive andnegative item indicators at 1312. The AR generator is an AR device forgenerating AR images, such as, but not limited to, the AR generatordevice 136 and the AR generator 144 in FIG. 1. The process terminatesthereafter.

Returning to 1304, if there are no high-interest items in the pluralityof items, the item filter component determines whether there are anylow-interest items at 1308. If no, the AR generator generates the ARdisplay at 1312. The process terminates thereafter.

While the operations illustrated in FIG. 13 are performed by a computingdevice, aspects of the disclosure contemplate performance of theoperations by other entities. In some examples, a cloud service performsone or more of the operations.

FIG. 14 is an exemplary flow chart illustrating operation of thecomputing device to generate a graphical representation of un-stockeditems in an AR display. The process shown in FIG. 14 can be performed bya customized AR filter component 126, executing on a computing device,such as the computing device 102 or the user device 116 in FIG. 1.

The process begins by determining whether a location of a user device isidentified at 1402. The user device includes any type of computingdevice including an AR generator, such as, but not limited to, the userdevice 116 in FIG. 1, the user device 202 in FIG. 2, and/or the userdevice 204 in FIG. 2. The location of the user device can be determinedbased on an analysis of sensor data generated by sensor devicesassociated with the user device and/or the item selection area.

If yes, the customized AR filter component identifies a plurality ofitems within a predetermined range of user device at 1404. Thepredetermined range is a distance or zone associated with the userdevice, such as, but not limited to, the predetermined distance 652 inFIG. 6.

The customized AR filter component identifies high-interest items in theplurality of items at 1406. The customized AR filter componenthighlights high-interest items in an AR display at 1408. The customizedAR filter component identifies low-interest items at 1410. Thecustomized AR filter component overlays low-interest with items withnegative item indicators at 1412. The customized AR filter componentdetermines if any un-stocked items of interest at 1414. If no, theprocess terminates thereafter.

Returning to 1414, if there are un-stocked items predicted to be ofinterest to the user, the customized AR filter component generates agraphical representation of the un-stocked items in AR display at 1416.The process terminates thereafter.

While the operations illustrated in FIG. 14 are performed by a computingdevice, aspects of the disclosure contemplate performance of theoperations by other entities. In some examples, a cloud service performsone or more of the operations.

FIG. 15 is an exemplary flow chart illustrating operation of thecomputing device to update an AR display in real-time. The process shownin FIG. 15 can be performed by a customized AR filter component,executing on a computing device, such as the computing device 102 or theuser device 116 in FIG. 1.

The process begins by generating an AR display via an AR generator at1502. The customized AR filter component determines whether apredetermined time has passed at 1504, such as the customized AR filtercomponent 126 in FIG. 1, FIG. 6 and FIG. 9. If no, the customized ARfilter component waits until the predetermined time is passed at 1506.The customized AR filter component obtains updated image data at 1508.The updated image data is obtained from sensor devices, such as the oneor more image capture device(s) 1102 in FIG. 11. The customized ARfilter component updates the AR display at 1510. The customized ARfilter component determines if user input is received at 1512. The userinput can include input such as, but not limited to, the user input 634in FIG. 6.

If user input is not received, the process returns to 1504. The processiteratively executes operations 1504 through 1512 until user input isreceived at 1512. If user input is received at 1512, the customized ARfilter component updates the AR display based on the user input at 1514.In some examples, the AR display is updated at a predetermined timeinterval, such as, but without limitation, the predetermined timeinterval 650 in FIG. 6.

In other examples, the AR display is updated/reset from anywhere withinthe item selection area. In other words, as the user moves aroundthroughout the item selection area, the AR display is updated to reflectthe changing FOV of the user/user device.

The customized AR filter component determines whether to continue at1516. If yes, the process returns to 1504 and iteratively executesoperations 1504 through 1516 until a determination is made not tocontinue. In some examples, the determination not to continue is madewhen the user exits the item selection area and/or completes atransaction associated with at least one item. The process terminatesthereafter.

While the operations illustrated in FIG. 15 are performed by a computingdevice, aspects of the disclosure contemplate performance of theoperations by other entities. In some examples, a cloud service performsone or more of the operations.

FIG. 16 is an exemplary flow chart illustrating operation of thecomputing device to apply customized item filters via AR. The processshown in FIG. 16 can be performed by a customized AR filter component,executing on a computing device, such as the computing device 102 or theuser device 116 in FIG. 1.

The customized AR filter component determines a location of a userdevice within an item selection area at 1602. The customized AR filtercomponent loads initial data at 1604. The initial data can include itemimage data, item markers, item data, aisle location data, planogramdata, item selection criteria, etc.

The customized AR filter component obtains image data of real-worldwithin the FOV of the user device at 1606. The customized AR filtercomponent determines whether there are any items within the FOV of theuser device at 1608. If there are no items within the FOV of the userdevice, the process terminates thereafter.

If there are any items within the FOV of the user device at 1608, thecustomized AR filter component applies the applicable item filters at1610. The customized AR filter component outputs the AR display at 1612.The customized AR filter component determines if there are any supplierincentives at 1614. Incentives can include promotionscustomized/specific to the user, such as, but not limited to, thepromotional offer 906 in FIG. 9. If yes, the customized AR filtercomponent un-filters any low-interest items associated with the supplierincentives at 1616. The process terminates thereafter.

If there are no supplier incentives at 1614, the customized AR filtercomponent determines if there is a user lookup at 1618. The user lookupis a request for additional information from the user, such as, but notlimited to, the user input 634 in FIG. 6. If no, the process terminatesthereafter.

Returning to 1618, if the customized AR filter component receives anitem lookup, the customized AR filter component outputs links and/orother additional information at 1620. The information is informationassociated with one or more high-interest items, such as, but notlimited to, the additional information 632 in FIG. 6. The processterminates thereafter.

While the operations illustrated in FIG. 16 are performed by a computingdevice, aspects of the disclosure contemplate performance of theoperations by other entities. In some examples, a cloud service performsone or more of the operations.

ADDITIONAL EXAMPLES

In some examples, as a user walks down an aisle in an item selectionarea, the customized AR filter component determines a location of theuser device and applies selection criteria/applicable filters tohighlight items of interest, obscure low-interest items, and/or generategraphical representations of out-of-stock high-interest items availablefor restock or order. The user can select items for order by touching ascreen on the user device, gesturing, speaking a voice command/request,touching a three-dimensional item on the scree/in the AR display,touching a real-world item, reaching for an item, clicking on an item,or providing another user input. This enables an individual (customer orassociate) to use AR to highlight or only show, items that have certainqualities desired by the individual. In some examples, the systemfilters for new items, items with certain ingredients, items thatachieve a specific purpose, etc.

In an exemplary illustration, a user opens the filter application on auser device as the user enters a store. A customized AR filter componenton a server or other computing device receives sensor data from the userdevice to determine a location of the user. The customized AR filtercomponent identifies a plurality of items assigned to the area within apredetermined range of the user. The customized AR filter componentobtains the user's selection criteria/search terms and item data,including item attributes for the user. The customized AR filtercomponent obtains image data from the user device. The image data isanalyzes using image recognition to identify items in the FOV of theuser or user device to determine which items the user is looking atusing markers/item recognition, etc. The customized AR filter componentperforms filtering on the items in the FOV to identify high-interest andlow-interest items within the FOV as well as items outside theFOV/un-stocked items which can also be high-interest based on thetype/category and filter criteria. Low-interest items are greyed out orobscured. If the user provides additional input, such as providingverbal queries, reaching for items, touching a graphical representationof an item on a screen, or other user input, the system responds to theuser input. The response can include fine-tuning the search/filterresults, outputting additional information, providing orderinginformation, outputting a link to another website, etc.

In another example, the AR system filtering products includes sensordevice(s) that capture image data of items in real-time. The systemutilizes AR technique to obtain image data. The system considers userpreferences, user input entered by user in real-time and/or filtercriteria. The system utilizes the user data and image data to filter theproducts according to user preference/requirements. The system utilizesAR to highlight/show the filtered products according to the filteredcriteria of user.

The system in one example filters for brand line products in samefeature set groups of products a user can purchase/use together. Thebrand-line products include items in the same line of items, same brand,and/or same grouping. The system can also filter for items associatedtogether but not same brand. For example, high-definition multimediainterface (HDMI) cable with a television.

In another example, feedback includes item feedback for itemrepurchases. If a user purchases the same item or type of item two ormore items, the system can output a feedback request to the user. Theuser can be provided with a reward/incentive for providing the feedback.

Cryptocurrency can be accepted in some examples to complete atransaction associated with ordering an un-stocked virtual item (ghostproduct). The system can provide additional purchase options, such asdiscounts/rewards if the user removes/rejects certain options, such asreturn and exchange options. In another example, the systemprovides/offers the user with a lower price/reduced shipping costs ifthe user opts to group an item with other ordered items and/or agrees topick-up an item at a store instead of delivery of the item to the user'sresidence. If the user opts out, the system can offer/provide the userwith a discount/reduced purchase price for the item. These transactionsordering items and/or purchasing items via online methods, can beperformed in the store at the shelf.

The un-stocked virtual items filtered for the user can include itemsprovided by local businesses/online marketplace via different retailersspecific to the item selection area's geographic region. If the userorders these un-stocked items, the user can opt to pick-up the items atthe item selection area at a later item.

In an illustrative example, a user rearranges an augmented reality imageof the modular display to suit their preferences. The user can select agraphical representation associated with an item in the AR display andmove the item to a different location within the AR display to alter thearrangement of items to suit the user's personal preference. The changeto the display can be a one-time only change or a permanent change/newdefault arrangement for the items in the selected user's preferences. Inother words, the changes the user makes to the arrangement of graphicalrepresentations of items in the AR display become the new defaultarrangement customized for that user.

In other examples, the user can alter the AR display to suit the user'spreferences by changing color of one or more shelves, adding virtualsignage, adding a graphical character, or other graphical designs to theAR display. For example, a user can alter the AR display to include agraphical unicorn character which moves throughout the AR display. Inanother example, a user can alter the AR display to include graphicalimages of birds, squirrels, cats, or other animals moving throughout theAR display. Another user can alter the AR display to add a flowerpattern to shelfs, change color of walls or floors, etc.

The system in some examples outputs a query to the user regarding theuser's current location within an item selection area. For example, thefilter application can output a list of item areas, such as, dairy area,toy department, sports equipment, pet care area, etc. The user selectsthe area of the store where the user is currently located. The userdevice sends the selected area to the filter component. The filtercomponent utilizes the user-designated area to begin identifying itemswithin the predetermined range of the user (FOV of the user) and applyselection criteria to those items for AR filtering.

The system can also output content to electronic/digital signs. Thecontent can include product images, price data, item information, etc.This can be useful where the user device is a cell phone or other userdevice having a small screen. Syncing the AR display content data withthe electronic/digital output device (display screen on shelves),enables easier viewing of content by user. The content can be sentthrough firewall webservice to the digital output device to change theelectronic sign.

Alternatively, or in addition to the other examples described herein,examples include any combination of the following:

-   a set of virtual signs within the AR display, the set of virtual    signs comprising information associated with at least one item in    the set of high-interest items or at least one item in the set of    un-stocked virtual items;-   an un-stocked virtual items manager, implemented on the at least one    processor, wherein the graphical representation of each item in the    set of un-stocked virtual items comprises a status indicator;-   wherein the status indicator comprises at least one of an alternate    location indicator associated with items currently available in    another display at a different location within the item selection    area;-   a re-stock indicator associated with items available in current    inventory for re-stocking an empty item display upon request;-   an order indicator associated with items available for order from a    remote location for pick-up by the user or delivery to the user at a    future date;-   a pop-up generator, implemented on the at least one processor, that    outputs additional information associated with at least one item in    the set of high-interest items within the AR display in response to    receiving a user input associated with the at least one item, the    additional information comprising at least one or item data, online    purchase information, or a link to a website associated with at    least one item;-   a feedback component, implemented on the at least one processor,    that outputs a request for feedback associated with an arrangement    of items within the AR display or feedback associated with an item    within the AR display selected for purchase by the user;-   a promotions component, implemented on the at least one processor,    that receives at least one promotional offer associated with an item    in the set of low-interest items from a remote computing device    associated with a supplier in real-time;-   wherein the filter component further analyzes the promotional offer    using the set of weighted selection criteria and the set of user    preferences to determine whether to move the item from the set of    low-interest items to the set of high-interest items;-   wherein the filter component moves the promotional item from the set    of low-interest items to the set of high-interest items on condition    the set of weighted selection criteria and the set of user    preferences indicates the promotional offer increases potential    interest of the user in the promotional item;-   an item status update received from the filter component, by the    overlay update component, identifying the promotional item as a    high-interest item;-   wherein the AR generator generates an updated AR display in    real-time;-   wherein the updated AR display comprises a positive item indicator    associated with the item, wherein the negative item indicator    associated with the promotional item is removed from the updated AR    display;-   a virtual shelf within the AR display, the virtual shelf comprising    at least one graphical representation of at least one un-stocked    virtual item predicted to be of interest to the user based on    user-provided data and historical transaction data associated with    the user;-   a re-stock control associated with an item in the set of un-stocked    items, wherein a notification is sent to an associate to re-stock an    item in response to a user selection of the re-stock control in    response to a determination at least one instance of the item in the    set of un-stocked items is available in a storage area;-   an item-order request control associated with an item in the set of    un-stocked items, wherein an order component orders the un-stocked    item for pick-up by the user or delivery to an address provided by    the user in response to user selection of the item-order request    control and completion of an item order;-   a streaming content display node within the AR display, the    streaming content associated with at least one item in the set of    high-interest items or at least one user preference within a set of    user AR display preferences;-   receiving, by a promotions component, a promotional offer associated    with at least one item in the set of low-interest items;-   analyzing, by a filter component, the promotional offer using a set    of per-user selection criteria;-   on condition the filter component determines the at least one item    is of potential interest to the user based on the analysis of the    promotion offer, dynamically removing, by the AR generator, a    negative item indicator associated with the at least one item;-   outputting the updated AR display to the user, the updated AR    display comprising at least one positive item indicator associated    with the at least one item;-   analyzing, by a feedback component, feedback associated with the set    of high-interest items;-   updating the set of per-user selection criteria based on the    analysis of the feedback;-   sending a notification to a restock manager requesting re-stock of a    selected un-stocked virtual item in response to user selection of a    re-stock button associated with the selected un-stocked item within    the AR display;-   outputting an order request to the user on condition the user    activates an item-order request control within the AR display via a    user interface associated with the user device, wherein a completed    order request is transmitted to a remote computing device by the    user device to place an order requesting an un-stocked item be    delivered to an address provided by the user or picked-up by the    user at a predetermined time;-   modifying contents of the AR display in response to receiving user    input altering at least one graphical element within the AR display,    wherein modifying the content of the AR display comprises modifying    an arrangement of at least one item within the AR display or    altering an appearance of at least one item within the AR display;-   identifying a digital output device within a predetermined distance    of the user device, the digital output device attached to a physical    set of shelves associated with the plurality of items;-   transmitting customized content from the user device to the digital    output device for display to the user, wherein the customized    content comprises information associated with at least one item in    the set of high-interest items or at least one item in the set of    un-stocked virtual items;-   analyzing item data using a set of weighted selection criteria and    the set of user preferences to identify a high-interest category of    items associated with the user, wherein the AR display comprises a    positive category indicator associated with all items in the    high-interest category within the FOV of the user;-   analyzing item data using a set of weighted selection criteria and    the set of user preferences to identify a low-interest category of    items associated with the user, wherein the AR display comprises a    negative category indicator associated with all items in the    low-interest category within the FOV of the user;-   receiving a selection of an item within the AR display;-   identifying a set of related items, wherein the set of related items    comprises at least one of an item in a same brand-line as the    selected item or at least one item associated with the selected    item;-   updating the AR display to include a graphical representation of the    set of related items;-   a prediction component, implemented on the at least one processor,    that identifies a set of un-stocked items physically absent from the    predetermined range of the user of potential interest to the user,    wherein the AR display comprises a graphical representation of each    item in the set of un-stocked items within a virtual item display;    and-   an un-stocked items status indicator comprising at least one of an    alternate location indicator associated with items currently    available in another display at a different location within the item    selection area;-   a re-stock indicator associated with items available in current    inventory for re-stocking an empty item display upon request; and/or-   an order indicator associated with items available for order from a    remote location for pick-up by the user or delivery to the user at a    future date.

At least a portion of the functionality of the various elements in FIG.1, FIG. 2, FIG. 3, FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, FIG. 9, FIG.10, FIG. 11, and FIG. 12 can be performed by other elements in FIG. 1,FIG. 2, FIG. 3, FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, FIG. 9, FIG. 10,FIG. 11, and FIG. 12, or an entity (e.g., processor 106, web service,server, application program, computing device, etc.) not shown in FIG.1, FIG. 2, FIG. 3, FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, FIG. 9, FIG.10, FIG. 11, and FIG. 12.

In some examples, the operations illustrated in FIG. 13, FIG. 14, FIG.15 and FIG. 16 can be implemented as software instructions encoded on acomputer readable medium, in hardware programmed or designed to performthe operations, or both. For example, aspects of the disclosure can beimplemented as a system on a chip or other circuitry including aplurality of interconnected, electrically conductive elements.

While the aspects of the disclosure have been described in terms ofvarious examples with their associated operations, a person skilled inthe art would appreciate that a combination of operations from anynumber of different examples is also within scope of the aspects of thedisclosure.

The term “Wi-Fi” as used herein refers, in some examples, to a wirelesslocal area network using high frequency radio signals for thetransmission of data. The term “BLUETOOTH®” as used herein refers, insome examples, to a wireless technology standard for exchanging dataover short distances using short wavelength radio transmission. The term“cellular” as used herein refers, in some examples, to a wirelesscommunication system using short-range radio stations that, when joinedtogether, enable the transmission of data over a wide geographic area.The term “NFC” as used herein refers, in some examples, to a short-rangehigh frequency wireless communication technology for the exchange ofdata over short distances.

While no personally identifiable information is tracked by aspects ofthe disclosure, examples have been described with reference to datamonitored and/or collected from the users. In some examples, notice canbe provided to the users of the collection of the data (e.g., via adialog box or preference setting) and users are given the opportunity togive or deny consent for the monitoring and/or collection. The consentcan take the form of opt-in consent or opt-out consent.

Exemplary Operating Environment

Exemplary computer readable media include flash memory drives, digitalversatile discs (DVDs), compact discs (CDs), floppy disks, and tapecassettes. By way of example and not limitation, computer readable mediacomprise computer storage media and communication media. Computerstorage media include volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules and the like. Computer storage media are tangible andmutually exclusive to communication media. Computer storage media areimplemented in hardware and exclude carrier waves and propagatedsignals. Computer storage media for purposes of this disclosure are notsignals per se. Exemplary computer storage media include hard disks,flash drives, and other solid-state memory. In contrast, communicationmedia typically embody computer readable instructions, data structures,program modules, or the like, in a modulated data signal such as acarrier wave or other transport mechanism and include any informationdelivery media.

Although described in connection with an exemplary computing systemenvironment, examples of the disclosure are capable of implementationwith numerous other general purpose or special purpose computing systemenvironments, configurations, or devices.

Examples of well-known computing systems, environments, and/orconfigurations that can be suitable for use with aspects of thedisclosure include, but are not limited to, mobile computing devices,personal computers, server computers, hand-held or laptop devices,multiprocessor systems, gaming consoles, microprocessor-based systems,set top boxes, programmable consumer electronics, mobile telephones,mobile computing and/or communication devices in wearable or accessoryform factors (e.g., watches, glasses, headsets, or earphones), networkPCs, minicomputers, mainframe computers, distributed computingenvironments that include any of the above systems or devices, and thelike. Such systems or devices can accept input from the user in any way,including from input devices such as a keyboard or pointing device, viagesture input, proximity input (such as by hovering), and/or via voiceinput.

Examples of the disclosure can be described in the general context ofcomputer-executable instructions, such as program modules, executed byone or more computers or other devices in software, firmware, hardware,or a combination thereof. The computer-executable instructions can beorganized into one or more computer-executable components or modules.Generally, program modules include, but are not limited to, routines,programs, objects, components, and data structures that perform tasks orimplement particular abstract data types. Aspects of the disclosure canbe implemented with any number and organization of such components ormodules. For example, aspects of the disclosure are not limited to thespecific computer-executable instructions or the specific components ormodules illustrated in the figures and described herein. Other examplesof the disclosure can include different computer-executable instructionsor components having more or less functionality than illustrated anddescribed herein.

In examples involving a general-purpose computer, aspects of thedisclosure transform the general-purpose computer into a special-purposecomputing device when configured to execute the instructions describedherein.

The examples illustrated and described herein as well as examples notspecifically described herein but within the scope of aspects of thedisclosure constitute exemplary means for customized AR item filtering.For example, the elements illustrated in FIG. 1, FIG. 2, FIG. 3, FIG. 4,FIG. 5, FIG. 6, FIG. 7, FIG. 8, FIG. 9, FIG. 10, FIG. 11, and FIG. 12,such as when encoded to perform the operations illustrated in FIG. 13,FIG. 14, FIG. 15 and FIG. 16, constitute exemplary means for identifyinga location of a user device associated with a user and a plurality ofitems within a predetermined distance of the user device based on theidentified location and a site-specific planogram; exemplary means foridentifying a set of high-interest items and a set of low-interest itemsfrom the plurality of items; exemplary means for identifying a set ofun-stocked items of potential interest to the user, each item in the setof un-stocked items is physically absent from a set of displays withinthe FOV of the user; exemplary means for assigning a positive itemindicator to each item in a set of high-interest items, a negative itemindicator for each item in the set of low-interest items, and agraphical representation for each item in the set of un-stocked items;and exemplary means for generating an AR display of a portion of an itemselection area within a FOV of the user to the user device for displayto the user.

The order of execution or performance of the operations in examples ofthe disclosure illustrated and described herein is not essential, unlessotherwise specified. That is, the operations can be performed in anyorder, unless otherwise specified, and examples of the disclosure caninclude additional or fewer operations than those disclosed herein. Forexample, it is contemplated that executing or performing a particularoperation before, contemporaneously with, or after another operation iswithin the scope of aspects of the disclosure.

When introducing elements of aspects of the disclosure or the examplesthereof, the articles “a,” “an,” “the,” and “said” are intended to meanthat there are one or more of the elements. The terms “comprising,”“including,” and “having” are intended to be inclusive and mean thatthere can be additional elements other than the listed elements. Theterm “exemplary” is intended to mean “an example of.” The phrase “one ormore of the following: A, B, and C” means “at least one of A and/or atleast one of B and/or at least one of C.”

Having described aspects of the disclosure in detail, it will beapparent that modifications and variations are possible withoutdeparting from the scope of aspects of the disclosure as defined in theappended claims. As various changes could be made in the aboveconstructions, products, and methods without departing from the scope ofaspects of the disclosure, it is intended that all matter contained inthe above description and shown in the accompanying drawings shall beinterpreted as illustrative and not in a limiting sense.

The embodiments of the invention in which an exclusive property orprivilege is claimed are defined as follows:
 1. An augmented realitysystem for filtering items in real-time, the system comprising: amemory; at least one processor communicatively coupled to the memory; aset of sensor devices generating sensor data associated with a pluralityof items within a field of view (FOV) of a user device associated with auser in an item selection area; an item filter component, implemented onthe at least one processor, that analyzes item data associated with theplurality of items and the sensor data using a set of weighted selectioncriteria and a set of user preferences to select a set of high-interestitems and a set of low-interest items for the user, the set ofhigh-interest items comprising a set of items physically availablewithin the FOV of the user device and a set of items physically absentfrom the FOV of the user device; an overlay component, implemented onthe at least one processor, that assigns a positive item indicator toeach item in the set of high-interest items and a negative itemindicator for each item in the set of low-interest items; and anaugmented reality (AR) generator, implemented on the at least oneprocessor, that outputs an AR display of the FOV of the user device, theAR display comprising a real-world image of a portion of the itemselection area within the FOV of the user and an AR overlay, the ARoverlay comprising a set of positive item indicators associated witheach item in the set of high-interest items and a set of negative itemindicators associated with each item in the set of low-interest items,and a virtual display comprising a set of un-stocked virtual itemsphysically absent from the FOV of the user device.
 2. The system ofclaim 1, further comprising: a set of virtual signs within the ARdisplay, the set of virtual signs comprising information associated withat least one item in the set of high-interest items or at least one itemin the set of un-stocked virtual items.
 3. The system of claim 1,further comprising: an un-stocked virtual items manager, implemented onthe at least one processor, wherein a graphical representation of eachitem in the set of un-stocked virtual items comprises a statusindicator, wherein the status indicator comprises at least one of analternate location indicator associated with items currently availablein another display at a different location within the item selectionarea, a re-stock indicator associated with items available in currentinventory for re-stocking an empty item display upon request, and anorder indicator associated with items available for order from a remotelocation for pick-up by the user or delivery to the user at a futuredate.
 4. The system of claim 1, further comprising: a pop-up generator,implemented on the at least one processor, that outputs additionalinformation associated with at least one item in the set ofhigh-interest items within the AR display in response to receiving auser input associated with the at least one item, the additionalinformation comprising at least one of online order information or alink to a website associated with at least one item.
 5. The system ofclaim 1, further comprising: a feedback component, implemented on the atleast one processor, that outputs a request for feedback associated withan arrangement of items within the AR display or feedback associatedwith an item within the AR display selected for purchase by the user. 6.The system of claim 1, further comprising: a promotions component,implemented on the at least one processor, that receives a promotionaloffer associated with a first item in the set of low-interest items froma remote computing device associated with a supplier in real-time,wherein the filter component further analyzes the promotional offerusing the set of weighted selection criteria and the set of userpreferences to determine whether to move the item from the set oflow-interest items to the set of high-interest items, wherein the filtercomponent moves the first item from the set of low-interest items to theset of high-interest items on condition the set of weighted selectioncriteria and the set of user preferences indicates the promotional offerincreases potential interest of the user in the item.
 7. The system ofclaim 6, further comprising: an item status update received from thefilter component, by the overlay update component, identifying the firstitem as a high-interest item, wherein the AR generator generates anupdated AR display in real-time, and wherein the updated AR displaycomprises a positive item indicator associated with the item, whereinthe negative item indicator associated with the first item is removedfrom the updated AR display.
 8. The system of claim 1, furthercomprising: a virtual shelf associated with the virtual display withinthe AR display, the virtual shelf comprising at least one graphicalrepresentation of at least one un-stocked item predicted to be ofinterest to the user based on user-provided data and historicaltransaction data associated with the user.
 9. The system of claim 1,wherein the AR display further comprises: a re-stock control associatedwith an un-stocked item, wherein a notification is sent to an associateto re-stock an item in response to a user selection of the re-stockcontrol in response to a determination at least one instance of theun-stocked item is available in a storage area; and an item-orderrequest control associated with the un-stocked item, wherein an ordercomponent places un-stocked item on order for pick-up by the user ordelivery to an address provided by the user in response to the userselection of the item-order request control and completion of an itemorder request form.
 10. The system of claim 1, wherein the AR displayfurther comprises: a streaming content display node providing streamingcontent within the AR display, the streaming content associated with atleast one item in the set of high-interest items or at least one userpreference within the set of user preferences.
 11. Acomputer-implemented method for augmented reality item filtering, thecomputer-implemented method comprising: identifying, by an analysiscomponent, a location of a user device associated with a user and aplurality of items within a predetermined distance of the user devicebased on the identified location and a site-specific planogram;identifying, by an item filter component, a set of high-interest itemsand a set of low-interest items from the plurality of items;identifying, by a prediction component, a set of un-stocked items ofpotential interest to the user, each item in the set of un-stocked itemsis physically absent from a set of displays within the a of the user;assigning, by an overlay component, a positive item indicator to eachitem in a set of high-interest items, a negative item indicator for eachitem in the set of low-interest items, and a graphical representation ofeach item in the set of un-stocked items; and generating, by an ARdisplay generator, an AR display of a portion of an item selection areawithin a FOV of the user device for display to the user, the AR displaycomprising a real-world image of a portion of the item selection areawithin the FOV of the user device generated by the image capture deviceenhanced by an AR overlay, the AR overlay comprising a set of positiveitem indicators associated with each item in the set of high-interestitems, a set of negative item indicators for each item in the set oflow-interest items, and the graphical representation of each item in theset of un-stocked items.
 12. The computer-implemented method of claim11, further comprising: receiving, by a promotions component, apromotional offer associated with a first item in the set oflow-interest items; analyzing, by a filter component, the promotionaloffer using a set of per-user selection criteria; on condition thefilter component determines the at least one item is of potentialinterest to the user based on the analysis of the promotion offer,dynamically removing, by the AR generator, a first negative itemindicator associated with the first item; and outputting an updated ARdisplay to the user, the updated AR display comprising at least onepositive item indicator associated with the first item.
 13. Thecomputer-implemented method of claim 11, further comprising: analyzing,by a feedback component, feedback associated with the set ofhigh-interest items; and updating the set of per-user selection criteriabased on the analysis of the feedback.
 14. The computer-implementedmethod of claim 11, wherein the AR display comprises a restock control,and further comprising: sending a notification to a restock managerrequesting re-stock of a selected un-stocked virtual item in response touser selection of a re-stock button associated with the selectedun-stocked item within the AR display; outputting an order request tothe user on condition the user activates an item-order request controlwithin the AR display via a user interface associated with the userdevice, wherein a completed order request is transmitted to a remotecomputing device by the user device to place an order requesting anun-stocked item be delivered to an address provided by the user orpicked-up by the user at a predetermined time; and modifying contents ofthe AR display in response to receiving user input altering at least onegraphical element within the AR display, wherein modifying the contentof the AR display comprises modifying an arrangement of at least oneitem within the AR display or altering an appearance of at least oneitem within the AR display.
 15. The computer-implemented method of claim11, further comprising: identifying a digital output device within thepredetermined distance of the user device, the digital output deviceattached to a physical set of shelves associated with the plurality ofitems; and transmitting customized content from the user device to thedigital output device for display to the user, wherein the customizedcontent comprises information associated with at least one item in theset of high-interest items or at least one item in a set of un-stockedvirtual items.
 16. The computer-implemented method of claim 11, furthercomprising: analyzing item data using a set of weighted selectioncriteria and a set of user preferences to identify a high-interestcategory of items associated with the user, wherein the AR displaycomprises a positive category indicator associated with all items in thehigh-interest category within the FOV of the user device; and analyzingthe item data using the set of weighted selection criteria and the setof user preferences to identify a low-interest category of itemsassociated with the user, wherein the AR display comprises a negativecategory indicator associated with all items in the low-interestcategory within the FOV of the user device.
 17. The computer-implementedmethod of claim 11, further comprising: receiving a selection of an itemwithin the AR display; identifying a set of related items, wherein theset of related items comprises at least one of an item in a samebrand-line as the selected item or at least one item associated with theselected item; and updating the AR display to include the graphicalrepresentation of the set of related items within the AR display.
 18. Asystem for augmented reality item filtering, the system comprising: amemory; at least one processor communicatively coupled to the memory; ananalysis component, implemented on the at least one processor, thatanalyzes item data associated with a plurality of items within an itemselection area and a planogram to identify a location of a user withinthe item selection area and identify the plurality of items within apredetermined distance of the user; an item filter component,implemented on the at least one processor, that analyzes the pluralityof items using a set of selection criteria and user preferencesassociated with the user to select a high-interest category of items ofpotential interest to the user and a low-interest category of itemswithin the predetermined distance of the user; an overlay component,implemented on the at least one processor, that assigns a positivecategory indicator to each item in the selected high-interest categoryand a negative category indicator to each item in the selectedlow-interest category; an AR generator, implemented on the at least oneprocessor, that displays an AR display of a FOV of the user, the ARdisplay comprising a real-world image of a portion of the item selectionarea within the FOV of the user with an AR overlay enhancing thereal-world image, the AR overlay comprising a positive categoryindicator associated with each item in the selected high-interestcategory within the FOV and a negative category indicator for each itemin the selected low-interest category within the FOV, the AR generatorupdates the AR display in real-time to change an indicator associatedwith an identified item in the selected low-interest category from anegative category indicator to a positive category indicator oncondition a promotional offer associated with the identified item isreceived from a set of suppliers.
 19. The system of claim 18, furthercomprising: a prediction component, implemented on the at least oneprocessor, that identifies a set of un-stocked items physically absentfrom the predetermined distance that is predicted to be of potentialinterest to the user, wherein the AR display comprises a graphicalrepresentation of each item in the set of un-stocked items within avirtual item display; and an un-stocked items status indicatorassociated with each un-stocked item in the set of un-stocked items inthe AR display, an un-stocked items status indicator comprising at leastone of an alternate location indicator associated with items currentlyavailable in another display at a different location within the itemselection area, a re-stock indicator associated with items available incurrent inventory for re-stocking an empty item display upon request, oran order indicator associated with items available for order from aremote location for pick-up by the user or delivery to the user at afuture date.
 20. The system of claim 18, further comprising: a digitaloutput device receiving customized content from the user device fordisplay to the user, wherein the customized content comprisesinformation associated with at least one item in the set ofhigh-interest items or at least one item in a set of un-stocked virtualitems, wherein the digital output device displays the customized contentupon detection of the user device within a predetermined area associatedwith the digital output device, and wherein the digital output deviceresumes display of default content upon detection of an absence of theuser device within the predetermined distance.